Touch pressure input for devices

ABSTRACT

A computing device, such as a wearable device, may include at least two electrodes mounted on a body. The computing device may determine an electrical signal associated with a circuit that includes the at least two electrodes and the user. A pressure applied to at least one electrode of the at least two electrodes may be determined from the electrical signal, and at least one function of the computing device may be implemented, based on the pressure.

TECHNICAL FIELD

This description relates to device control using touch pressure input.

BACKGROUND

Computing devices, including computers, smartphones, and many types ofwearable devices, provide a large number and variety of features andfunctions to users. These features and functions are associated withcorresponding types of human interface components, which enable theusers to control the computing device in desired manners.

SUMMARY

According to a general aspect, a computing device may include aprocessor, a storage medium storing instructions, a body, and at leasttwo electrodes mounted on the body and in contact with a user of thecomputing device (when using the device properly, i.e., as providedfor). The instructions, when executed by the processor, cause thecomputing device to determine an electrical signal associated with acircuit that includes the at least two electrodes and the user,determine, from the electrical signal, a pressure applied to at leastone electrode of the at least two electrodes, and implement at least onefunction of the computing device, based on the pressure. Acomputer-implemented method may be executed using the computing device,and a computer program product may be stored on the storage medium tocause the computing device to execute the computer-implemented method.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features will beapparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of at least one computing device providing atouch pressure input.

FIG. 2 is a flowchart illustrating example operations of the system ofFIG. 1.

FIG. 3 illustrates a bottom view of a wearable device providing a touchpressure input, in accordance with example implementations of FIG. 1.

FIG. 4 illustrates a top view of a wearable device providing a touchpressure input, in accordance with example implementations of FIG. 1.

FIG. 5 illustrates a first use case example of the wearable device ofFIGS. 3A and 3B.

FIG. 6 illustrates a second use case example of the wearable device ofFIGS. 3A and 3B.

FIG. 7 illustrates example voltage signals that may be used to determinepressure, in accordance with example implementations of FIG. 1.

FIG. 8 illustrates example relationships between the voltage signals ofFIG. 7 and corresponding levels of pressure, in accordance with exampleimplementations of FIG. 1.

FIG. 9A is a flowchart illustrating detailed example operations of thewearable device of FIGS. 3-6, using the voltage signals of FIGS. 7 and8.

FIG. 9B is a block diagram of an example filtering operation of FIG. 9A.

FIG. 10A illustrates an example machine learning algorithm that may beused to related a voltage signal with motion artifacts to a touchpressure value, in accordance with example implementations of FIGS. 1and 9A.

FIG. 10B is a block diagram of an example implementation of aconvolutional neural regressor of FIG. 10A

FIG. 10C is a table illustrating example parameters of the example ofFIG. 10B

FIG. 11 is an example illustration of the system of FIG. 1, implementedusing earbud devices.

FIG. 12 is an example illustration of the system of FIG. 1, implementedusing smartglasses.

FIG. 13A is an example illustration of the system of FIG. 1, implementedusing a smartphone.

FIG. 13B is an example illustration of the system of FIG. 1, implementedusing a keyboard.

FIG. 14 shows an example of a computer device, mobile computer deviceand head mounted device according to at least one exampleimplementation.

DETAILED DESCRIPTION

Described systems and techniques enable touch pressure interactions ofusers with devices, including wearable devices, without requiring adedicated pressure sensor. For example, a wearable device may beoutfitted with at least two electrodes, positioned to enable the user tocomplete a closed circuit by touching the at least two electrodes. Theclosed circuit includes the at least two electrodes and the user, e.g.,at least a portion of the body of the user. The device, e.g., wearabledevice, is configured to determine a relative or absolute level ofpressure applied to at least one of the at least two electrodes touchedby the user when completing the closed circuit. In this way, the devicemay perform various types of functions in response to the determinedpressure, thereby providing the user with convenient and intuitive modesof interaction with the device.

Conventional techniques for implementing pressure-based device controlutilize or rely on one or more dedicated pressure sensors, such ascapacitive sensors, and/or sensors that attempt to measure a distancemoved by a pressed element (e.g., a button, or a portion of atouchscreen). Such pressure sensors may be relatively expensive and/orunreliable (e.g., may produce unreliable measurements, or otherwiseprone to malfunction). Moreover, such pressure sensors may beundesirably large, and it may be difficult or impossible to include suchpressure sensors on devices with small form factors, such as wearabledevices, or in any use case scenario in which space is at a premium.

In described implementations, however, no dedicated pressure sensor isrequired. Instead, an applied pressure is inferred or derived from anelectrical signal, e.g., from a voltage or frequency of the electricalsignal. For example, when the electrical signal includes components thatare motion artifacts reflecting the pressure applied by a userassociated with pressing an electrode, then described implementationscalculate and quantify an existence and extent of the degree of pressurebeing applied to the electrode. The calculated pressure may then be usedto control a device operation or other function of a device.

Consequently, described implementations enable touch pressureinteractions, using, e.g., simple, small, inexpensive electrodes thatmay be installed and used in conjunction with a wide variety ofcomputing devices and associated peripherals, including wearabledevices. Thus, desirable and advantageous features of pressure-basedcontrols may be obtained, including features that were previouslyimpractical or unavailable.

Moreover, such touch pressure control may be provided in conjunctionwith other types of control systems and actions, using the sameelectrodes. For example, a pair of electrodes may be used to providetouch pressure control as referenced above, and the same pair ofelectrodes may be used to provide an additional function, such asproviding electrocardiography (ECG) measurements.

FIG. 1 is a block diagram of at least one computing device 112 providinga touch pressure input. In the example of FIG. 1, a pressure detector102 is configured to determine touch pressure applied by a user 104 whenthe user establishes a closed circuit 106 by simultaneously contacting,e.g., touching (with the user's skin surface) a first electrode 108 anda second electrode 110 of the at least one computing device 112,including applying pressure to at least one of the first electrode 108and the second electrode 110. As referenced above, and described indetail below, the pressure detector 102 may be configured to determinean existence of this pressure using a measured electrical signal of theclosed circuit 106, and does not require a dedicated pressure sensor todo so.

The at least one computing device 112 may include any type of computingdevice that includes at least one processor 114, which may beconfigured, among other functions, to execute instructions stored usinga non-transitory computer-readable storage medium 116. A sensor 118 maybe used to measure desired properties of the electrical signal of theclosed circuit 106, such as current, voltage, or frequency.

As the at least one computing device 112 may represent many types ofcomputing devices, examples of which are provided herein, the many typesof input/output (I/O) hardware that may be used in such computingdevices are illustrated in the aggregate in FIG. 1 as I/O hardware 120.By way of non-limiting example, such I/O hardware may include a display(e.g., a touchscreen), a peripheral device (e.g., mouse, keyboard, orstylus), a button, a speaker or audio sensor, haptic sensor or output,or camera. In some implementations, the sensor 118, the first electrode108, and the second electrode 110 may be considered examples of the I/Ohardware, but are shown separately in FIG. 1 for ease of explanation offunctionality of the pressure detector 102.

In FIG. 1, the computing device 112 is illustrated as including a body113. The electrodes 108, 110 may be mounted on the body 113 in positionsaccessible to a skin surface of the user 104. Examples of the body 113may be understood from example implementations illustrated below, suchas in FIGS. 5, 6, and 11-13B.

In addition to the I/O hardware 120, the at least one computing device112 may execute and provide many types of software that utilizes, or isimplemented by, the at least one processor 114, the non-transitorycomputer-readable storage medium 116, and the I/O hardware 120. Suchsoftware (and associated functionality) is included and represented inFIG. 1 by a function manager 122.

That is, for purposes of the description of the simplified example ofFIG. 1, the function manager 122 should be understood to represent andencompass the implementation of any software that may be associated withoperation of the I/O hardware 120 and/or the pressure detector 102. Forexample, the function manager 122 may include an operating system andmany different types of applications that may be provided by the atleast one computing device 112, some examples of which are provided inmore detail, below.

In particular, in the examples of FIGS. 3A-5, the at least one computingdevice 112 may represent a wearable device, such as a smartwatch. Thefunction manager 122 may provide the function of ECG measurements forthe user 104, using signals from the first electrode 108 and the secondelectrode 110. Then, the same signals may be used at other times, whenthe ECG measurements are not being taken, to provide the types of touchpressure control described herein. In such cases, existing ECG-relatedelements of a smartwatch may be used to implement the touch pressurecontrol techniques described herein, without requiring any additionalhardware components to be added to the smartwatch.

More generally, in the example of FIG. 1, it may be observed that theclosed circuit 106 is established as including or traversing a body ofthe user 104. The ability to establish such a circuit has previouslybeen used in other contexts, such as the ECG context just referenced,because the resulting circuit and electrical signals thereof captureelectrical activities of, e.g., a heart of the user 104.

In contrast, however, described techniques seek to measure, quantify, orotherwise utilize motion artifacts of such electrical signals that aresuperimposed on the signal characteristics reflecting the electricalactivities of the heart of the user 104. In other words, as describedand illustrated in detailed examples below with respect to FIGS. 6-8,actions of the user 104 in applying pressure, e.g., to the firstelectrode 108 when the circuit 106 is closed, will cause correspondingdistortions of the electrical signal reflecting the electricalactivities of the heart of the user 104.

Then, the pressure detector 102 may be configured to measure suchdistortions, or motion artifacts, and relate the distortions to anexistence and extent of the pressure applied by the user 104 to thefirst electrode 108. For example, such relations may be determinedheuristically. In other examples, such as described below with respectto FIGS. 8 and 9, a trained machine learning model may be configured toclassify specific patterns in electrical signals of the closed circuit106 as corresponding to specific, pre-defined levels of pressure appliedby the user to one or both of the first electrode 108 and the secondelectrode 110.

For example, in FIG. 1, the pressure detector 102 is illustrated asincluding a filter 124. The filter 124 may be configured to removeportions of the electrical signal detected by the sensor 118, in orderto make the resulting, filtered signal more amenable to analysis by afeature mapper 126. For example, when the electrical signal contains aheart electrical activity component of the user 104, which may be adominant feature as compared to the motion artifacts utilized by thepressure detector 102, the filter 124 may be configured to remove thisheart electrical activity component.

Then, a feature mapper 126 may be configured to analyze the resulting,filtered signal, and map characteristics thereof to corresponding levelsof pressure applied by the user 104. For example, the feature mapper mayobtain spectral features from a voltage signal measured by the sensor118. Then, as just referenced, the feature mapper 126 may use either aheuristic or machine learning approach to mapping resulting spectralfeatures to levels of pressure applied by the user 104.

Once these pre-defined levels of pressure are determined, then afunction interface 128 may provide the determined levels of pressure tothe function manager 122. The function manager 122 may thus implementany preconfigured use of the detected pressure levels to operatingsoftware or other hardware of the at least one computing device 112.

For example, the determined pressure levels may be used to implementdiscrete functions, such as activating or deactivating a specificapplication or application feature, or selecting one application oranother for use. In other examples, the determined pressure levels maybe determined to finer levels of gradation, or as continuously-changingpressure levels. Such pressure levels may be used in correspondingcontexts that would benefit from this type of control, such asincreasing/decreasing a brightness a screen, or a volume of a speaker.

A feedback generator 130 may be configured to provide the user 104 withfeedback regarding a detection and characterization of the pressurebeing applied by the user 104, so that the user 104 may providenecessary levels or types of pressure, and obtain desired results withrespect to operating the at least one computing device 112. For example,the feedback generator 130 may leverage specific elements of the I/Ohardware 120 to provide a feedback signal of varying sorts. For example,the I/O hardware 120 may include a vibration generator, and the feedbackgenerator 130 may cause such a vibration generator to vibrate at levelsthat are directly proportional to pressure levels determined by thefeature mapper 126. In other examples, other types of feedback may beprovided for similar purposes, including audio and/or visual feedback.

In example implementations, one of the first electrode 108 and thesecond electrode 110 may be referred to as a passive electrode, which ismaintained in a default state of contact with the user 104 during anormal use of the at least one computing device 112. For example, whenthe at least one computing device 112 includes a smartwatch, as in theexamples of FIGS. 3A-5, below, the first electrode 108 may be providedon an underside of the smartwatch, and in contact with a wrist of theuser 104 during normal wear of the smartwatch. Then, the secondelectrode 110 may be provided as a button that is accessible forpressing by the user 104, in order to complete the closed circuit 106.

Thus, in example implementations, the circuit 106 may be maintained in adefault open state, and may be closed when the user 104 contacts, andapplies pressure to, the second electrode 110. In exampleimplementations in which the first electrode 108 is referred to as apassive electrode, the second electrode 110 may be referred to as anactive electrode. As described in more detail, below, three or moreelectrodes may be used, including two or more passive electrodes and/ortwo or more active electrodes.

Similar configurations of passive and active electrodes may beimplemented in other types of devices. For example, the at least onecomputing device 112 may be implemented as a pair of smart glasses, suchas virtual reality (VR) or augmented reality (AR) glasses. Then, thefirst electrode 108 may be positioned at a point on the glasses thatrests on, and is in contact with, a nose of the user 104, while thesecond electrode 110 may be positioned at one side (e.g., temple, orglasses arm) of the glasses.

Similar implementations may be provided with respect to earbuds. Forexample, one earbud may be provided with the first electrode 108contacting a user's ear when resting therein or thereon, while the otherearbud may have the second electrode 110 exposed for pressing by theuser 104.

In other types of implementations, both of the first electrode 108 andthe second electrode 110 may be in a default open state (e.g., may bothbe referred to as active electrodes), and the user 104 may be requiredto touch the first electrode 108 with one hand (e.g., the user's lefthand), and the second electrode 110 with the other hand (e.g., theuser's right hand). For example, the at least one computing device 112may represent a smartphone, and the user 104 may hold the smartphone inone hand, while contacting the first electrode 108 with that hand, andmay then apply pressure to the second electrode 110 with the other hand.

Similarly, in the smart glasses implementations referenced above, thefirst electrode 108 may be positioned at one side (e.g., temple, orglasses arm), while the second electrode 110 may be positioned at theopposite side of the glasses. Also, in the earbuds example, the firstelectrode 108 may be positioned at one earbud, while the secondelectrode 110 may be positioned at the other earbud.

As a result, in these and other example implementations, by applyingpressure to obtain desired functionality, the user 104 may be providedwith the feel of physically pushing an icon or other selected item of,or onto, a display of a device. Such a pushing feel may be enhanced bymaking the degree of pressure applied proportional to a speed ofreaction of the display. For example, when scrolling through a largenumber of icons (e.g., apps), the user may scroll faster by applyingmore pressure, or scroll slower by applying less pressure.

Thus, described implementations provide many features, advantages, uses,and functionalities that would be difficult, impractical, infeasible, orimpossible to achieve using conventional techniques for device control,including conventional pressure sensors. Moreover, users may be providedwith abilities to control devices in convenient and intuitive manners.

FIG. 2 is a flowchart illustrating example operations of the system ofFIG. 1. In the example of FIG. 2, operations 202-208 are illustrated asseparate, sequential operations. However, in various exampleimplementations, the operations 202-208 may be implemented in anoverlapping or parallel manner, and/or in a nested, iterative, looped,or branched fashion. Further, various operations or sub-operations maybe included, omitted, or substituted.

In FIG. 2, an electrical signal may be determined at a computing device,the electrical signal associated with a circuit that includes at leasttwo electrodes in contact with a user of the computing device, and auser (202). For example, the pressure detector 102 may determine theelectrical signal from the circuit 106 when the user 104 closes thecircuit 106 by contacting both the first electrode 108 and the secondelectrode 110. A voltage of the electrical signal may be measured,using, e.g., the sensor 118.

As described, the at least one computing device may include anycomputing device which may benefit from the types of touch pressureinput described herein, including laptops, tablet computers,smartphones, smartwatches, smart glasses, earbuds, and other wearabledevise. Other examples of types of devices may include cameras, kiosks,cooking appliances, and various other types of appliances.

In some implementations, one or more of the at least two electrodes maybe installed on a peripheral device, such as a keyboard, mouse, orstylus. In some such implementations, processing and storage hardwaremay be maintained on a separate device, in wired or wirelesscommunication with the peripheral device.

In the above and other types of example implementations, three or moreelectrodes may be used. For example, a single passive electrode may beused, and two or more active electrodes may be used. In this way, forexample, each of the two or more active electrodes may have a dedicatedfunction(s). Similarly, two passive electrodes and two active electrodesmay be provided. For example, in implementations using earbuds, eachearbud may have a passive and active electrode, so that the user 104 mayexperience one set of functionality when applying pressure to one activeelectrode in one ear (through establishing the circuit 106 with thepassive electrode in the opposite ear), while experiencing another setof functionality when applying pressure to the other active electrode inthe opposite ear (through establishing another instance of the circuit106 with the passive electrode in the first ear).

From the electrical signal, a pressure applied to at least one electrodeof the at least two electrodes may be determined (204). For example, thepressure detector 102, having determined the electrical signal using thesensor 118, may filter the received signal using the filter 124. Forexample, as described herein, the circuit 106 may include a largecomponent related to internal electrical activity of the user 104, suchas electrical heart activity of a heart of the user 104. Such componentsmay dominate, e.g., may be significantly larger than, motion artifactsresulting from pressure applied by the user 104 to the activeelectrodes.

By removing these components, the filter 124 may enable the featuremapper 126 to map features of the electrical signal to correspondingpressure levels more accurately. For example, a measured voltage signalmay be analyzed to determine corresponding frequency characteristics(which may be referred to herein as spectral features). Then, thefeature mapper 126 may map the determined spectral features tocorresponding pressure levels, using, e.g., previously-determinedheuristics, or a previously-trained machine learning model, as describedin more detail, below.

At least one function of the computing device may be implemented, basedon the pressure (206). For example, the function interface 128 maycommunicate with the function manager 122 to implement virtually anyavailable function of the at least one computing device 112.

As described herein, such functionality may be dedicated, such as whenpressing the second electrode 110 provides a dedicated, pre-configuredfunction. In other examples, the functionality may be configurable bythe user 104, such as when the user 104 can designate a desired functionto be performed in response to application of pressure to the secondelectrode 110. In some implementations, the functionality may vary bycontext. For example, the functionality providing in the context of oneapplication may be different than the functionality provided in anothercontext.

In some implementations, the at least two electrodes may provideadditional functionality, beyond the touch pressure input techniquesprovided herein. For example, the function manager 122 may provide anECG measurement function and related applications. When the at least onecomputing device 112 (e.g., smartwatch) is in an ECG mode, theelectrodes 108, 110 may be used to provide ECG measurements, related tothe electrical heart activity of the user 104. In such implementations,the motion artifacts used by the pressure detector 102 may be filteredout of the electrical signal measured using the first electrode 108 andthe second electrode 110, as such motion artifacts and any other signalcomponents may be considered to be noise for purposes of providing ECGmeasurements or other measurements related to the electrical heartactivity of the user 104.

Finally in FIG. 2, feedback may be provided to the user, characterizingthe pressure (208). For example, the feedback generator 130 may providevisual, audio, or haptic indicators to the user 104. For example, suchindicators may vary in direct proportion to an amount of pressureapplied to the second electrode 110. For example, visual indicators mayvary in brightness or displayed size, and audio or haptic indicators mayalso vary in extent in proportion to the determined pressure. Using suchfeedback, the user 104 may easily determine whether it is necessary toapply more or less pressure to achieve a desired result with respect tothe functionality being provided.

FIG. 3 illustrates a bottom view of a wearable device providing a touchpressure input, in accordance with example implementations of FIG. 1.FIG. 4 illustrates a top view of the wearable device of FIG. 3.Specifically, FIGS. 3 and 4 illustrate a smartwatch 302, which has afirst electrode 304 positioned on a surface of the smartwatch 302 thatwould typically be in contact with a skin surface of the user 104 whenthe smartwatch 302 is being worn by the user 104.

As referenced above, the electrode 304 may thus represent an example ofa passive electrode. As also illustrated, the electrode 304 may beprovided with a surface area that is as large as practicable relative toavailable surface area of the smartwatch 302. Accordingly, a reliabilityof a contact of the electrode 304 with a skin surface of the user 104may be increased.

Meanwhile, an electrode 306 may be implemented as a button or similarshape. That is, the electrode 306 may be a simple metal electrode, andit not required to have any separate or additional mechanicalfunctioning, such as a spring-loading mechanism. Nonetheless, as theelectrode 306 may provide a desired electrical contact in any shapedesired, the implementation of FIGS. 3 and 4 illustrate that theelectrode 306 may be implemented in a shape resembling a button for easeand familiarity of use of the user 104. More generally, as shown anddescribed with respect to the additional example implementations ofFIGS. 11-14, either of the electrodes 304, 306 may be sized, shaped, orpositioned in virtually any desired manner to achieve a desired userinteraction.

As shown in the top view of FIG. 4, the smartwatch includes a display402 that may provide any conventional or future features available inthe context of a smartwatch, including access to applications, messages,or other features or content. When seen from the top view of FIG. 4, itmay be observed that the user 104 is provided with the display 402 andthe electrode 306, and may operate the electrode 306 as apressure-sensitive button, without being required to take further actionwith respect to the electrode 304.

For example, in FIG. 5, the smartwatch 302 is illustrated as being wornon a wrist 502 of the user 104, and the electrode 306 is illustrated asbeing pushed or pressed by a finger 504 of the user 104. As alreadydescribed, the finger 504 contacting (e.g., pressing) the activeelectrode 306, while the passive electrode 304 is contacting the wrist502 (of the other arm of the user 104) completes the circuit 106 of FIG.1, so that an electrical signal traversing the body of the user may bedetermined and used to identify and characterize a pressure beingapplied to the active electrode 306.

In the example of FIG. 5, it is assumed that a low pressure click isapplied by the finger 504 and detected by the pressure detector 102 ofFIG. 1. Such a low pressure click may be configured to cause a functionrelated to an illustrated application 506 (such as a selection thereof).

In FIG. 6, a high pressure click may be applied, which may be configuredto cause a different function. For example, as shown, the high pressureclick of the active electrode 306 may cause the application 506 to slideto the left of the display 402, and may cause a second application 602to be displayed as available for selection. Consequently, the user 104may have a physical sensation or perception of pushing the application506 to the left to access or use the application 602, and consistentwith a direction of pressure of the finger 504 on the active electrode306.

Smartwatches are able to serve as a convenient gateway for manycommunication routines with connected smartphones, including, e.g.,notifications and text messaging. However, conventional smartwatcheslack the richness in human interaction mechanics, compared tosmartphones. For example, smartwatches typically do not have a largetouch display nor various sensors to fit in a small form factor.

Described techniques effectively provide a pressure click, in which anactive electrode, like the active electrode 306, provide a physicalbutton and associated software that is capable of sensing the pressureof a user's finger click, so that depending on the pressure level, anappropriate routine can be triggered. For example, as shown in FIGS. 5and 6, a level of external pressure applied by a click may be configuredto be directly proportional to a location in the display 402 in which anapps menu is being localized. As described, to the user 104, thenavigation through the apps menu may feel as though the apps 506, 602are being physically pushed to the side from one to next.

As also referenced above with respect to FIG. 1, someotherwise-conventional smartwatches have electrocardiography (ECG)modules that can measure sinus rhythms and even screen atrialfibrillation (AFib) for users interested in health monitoring. Forexample, the examples of FIGS. 3-6 may be used, in which a lead-I ECGtechnique is used with one electrode below the smartwatch touching thewrist skin and another electrode located at the watch crown, so thatwhen the user touches the crown with a finger of the other hand, aclosed circuit is formed between the smartwatch and the heart to measureelectrical activities of the heart.

In some example implementations, an existing ECG module in a smartwatchmay be leveraged to measure pressure levels for flexiblehuman-smartwatch interactions, as described herein. For example, in FIG.7, a first pressure 702 applied may result in a first signal 704, asecond pressure 706 applied may result in a second signal 708, and athird pressure 710 applied may result in a third signal 712. Forexample, the pressures 702, 706 and 706, 710 may vary by a factor ofabout 2-3.

As shown, heartbeats 714, 716, 718 may be detected in the resultingelectrical signals 704, 708, 712. Additionally, motion artifacts 720 maybe observed. As shown, the motion artifacts 720 are appreciably smallerthan the heartbeats 714, 716, 718, but increase (e.g., in magnitudeand/or frequency) in direct proportion to a degree of pressure appliedto the active electrode 306.

Thus, by observing the motion artifacts 720 that are present in the ECGsignals 704, 708, 712, a robust pressure sensing model may be generatedbased on the illustrated fact that different external pressures resultin different ECG motion artifact characteristics. Conventionally in thecontext of ECG measurements, such ECG motion artifacts are to becompletely avoided and rejected in hardware or in algorithms as they donot contain, or may obscure, useful information about the heartcondition that is desired. However, described techniques use thistraditional noise signal as a pressure input that enhanceshuman-smartwatch interactions by deviating away from its originalcardiovascular health use cases, and thereby opening up new, powerfulinteraction routines for a smartwatch user. For a smartwatch thatalready provides an ECG module, such interaction routines may berealized with little or no additional hardware, and merely by utilizingexisting ECG components.

In more detail, conventional smartwatches with ECG functionalityeffectively miniaturize an on-body lead-I ECG setup used in healthcaresettings, so that two electrodes that are clipped at two sides of thechest in the healthcare setting are ported to left and right hands of auser to obtain a closed circuit between the smartwatch and the user'sheart, such as illustrated in FIG. 1. The resulting ECG signal(s), suchas the signals 704, 708, 712, may be measured in millivolts and has aknown template (PQRST) for a heartbeat. This template containsinformation about the size and position of the heart chambers,illustrated by the heartbeat portions 714, 716, 718 of the signals 704,708, 712, respectively. As also described, the motion artifacts 720typically are considered to be contaminants in a conventional ECGmeasurement context. In described techniques, however, such motionartifacts contain and represent finger pressure applied by the finger onthe ECG electrode. The harder the press is, the noisier the raw ECGsignal becomes, and described techniques utilize this finger presssensitivity to infer click strength.

FIG. 8 illustrates example relationships between the voltage signals ofFIG. 7 and corresponding levels of pressure, in accordance with exampleimplementations of FIG. 1. As shown, a pressure signal 802 over time maybe generated which is inferred from, and correspond to, the signals 704,706, 708. That is, FIG. 8 illustrates an example of how output pressurecorresponds to different noise levels (including micromotion artifacts)of an ECG input signal.

As shown, the signal 704 relates to a first pressure level of thepressure signal 802, e.g., a low pressure touch. The signal 706 relatesto a second pressure level of the pressure signal 802, e.g., a mediumpressure touch. The signal 708 relates to a third pressure level of thepressure signal 802, e.g., a high pressure touch.

FIG. 9A is a flowchart illustrating detailed example operations of thewearable device of FIGS. 3-6, using the voltage signals of FIGS. 7 and8. In FIG. 9A, an ECG recording (902) of a raw ECG signal 904 may beobtained from a touch pressure input, such as described above withrespect to FIG. 5, e.g., when the user 104 touches the smartwatch at adesignated electrode.

In the context of ECG measurements, a QRS complex refers to deflectionsin an electrocardiogram (ECG) tracing, and represent ventricularactivity of a user's heart. That is, a QRS complex includes Q, R, and Swaves and represent a heart's electrical impulse spreading through theventricles of the user's heart. QRS waves are part of the PQRST templatefor a heart beat, and QRS notching (906) refers to adaptive filtering ofsuch electrical signals from the user's heart, using, e.g., an inversematched filter. In this way, a notched ECG stream 908 may be obtained.

The notched ECG stream 908 thus corresponds to a noise level signal, onwhich spectral feature extraction (910) may be performed to describefrequency contents and characteristics of the notched ECG stream 908. Inthis way, spectral features 912 are obtained.

FIG. 9B illustrates a more detailed example of the QRS notching 906 ofFIG. 9A. In FIG. 9B, a sliding window 918 scans through the measured ECGstream 904. Each window contains N samples of the ECG stream 904. Eachwindow independently goes through a wavelet thresholding procedure 920.Such wavelet thresholding works well with the ECG peak, which is(extremely) time-concentrated. Frequency-based filtering techniques mayalso be used, but may be less suitable to separate peak from non-peaksignal as well as the described example. That is, the describedwavelet-based techniques are more time-frequency neutral, and canaccomplish peak notching without shaving off the motion artifactsdescribed herein. Once the thresholding operation 920 is done, theoverlapping windows 922 are stitched back using an overlap-and-averageoperation 924 to represent a time series with QRS peaks notched out,shown as output, notched ECG stream 908.

A pressure model 916 may then be used to perform convolutionalregression (914) on the spectral features 912, to thereby obtain thepressure signal 802 of FIG. 8. That is, for example, a convolutionalneural network may be trained, using sufficient quantities of trainingdata, to accurately map the spectral feature 912 to the pressure signal802.

For example, tests may be run using one or more pressure sensors tomeasure actual pressure levels being applied by users, in conjunctionwith corresponding pressures applied to ECG electrodes. In this way, themicromotion artifacts of the measured ECG signals may be correlated withthe corresponding pressures.

For example, FIG. 10A illustrates an example machine learning algorithmthat may be used to relate a voltage signal with motion artifacts to atouch pressure value, in accordance with example implementations ofFIGS. 1,9A, and 9B. As shown in FIG. 10A, spectral features 1002 may bepassed through a convolutional neural network 1004 (including one ormore convolutional layers), and corresponding fully-connected layers1006, to obtain a corresponding touch pressure value 1008. That is, aconvolutional neural network may be used as a regression model todetermine the mapping from a spectral feature to a pressure level. Themodel may be used to provide a common set of weights during runtime forall users.

In particular, N may equal any value that may be obtained withsufficient quantity and quality of training, so that the user 104 may beprovided with a virtually continuous spectrum of pressure levels. In oneexample, a fully-connected layer 1006 takes an assigned input volumefrom a preceding layer, and outputs an N dimensional vector, where N isa number of classes available for assigning pressure levels. In otherwords, as referenced above with respect to FIG. 8, pressure levels maybe assigned to one, two, three, or more values, such as low, medium, orhigh. In the latter case, if N=3, then the three classes of low, medium,high may be available for designation.

In other examples, as also referenced, pressure levels may be determinedheuristically, without requiring the training and use of a machinelearning model. For example, voltage values and/or spectral features maybe associated with corresponding threshold levels, so that values abovea certain threshold are interpreted as high pressure touches, whilevalues below the threshold are interpreted as low pressure touches.Multiple thresholds may be used to obtain multiple pressure levels.

FIG. 10B is a block diagram of an example implementation of aconvolutional neural regressor of FIG. 10A. FIG. 10C is a tableillustrating example parameters of the example of FIG. 10B. In FIG. 10B,the spectral feature 912 of the notched ECG signal 908 enters aconvolutional neural network (such as the example convolutional neuralregressor 914 of FIG. 9A) to estimate a final pressure number. In anexample, if a frame rate for ECG is 500 Hz, the Nyquist theorem may beused to select a spectral feature with 250 Hz cap frequency. Fastfourier transform parameters may be selected to provide a frequencyfeature as a 250 dimensional vector 1010, with each frequency bincontaining amplitude information for a 1 Hz frequency slice.

The example convolutional neural network architecture of FIG. 10Billustrates two convolutional layers 1012, 1016, and average poolinglayers 1014, 1018, as well as two fully-connected layers 1020, 1022. Theconvolutional layers 1012, 1016 are configured to summarizespatially-coherent information hidden in the frequency domain feature1010. The pooling layers 1014, 1018 reduce a dimensionality (number ofparameters) of features present in regions of the convolutional layers1012, 1016. The fully-connected layers 1020, 1024, as described above,relate features of the mean pool layer 1018 to a particular class, sothat the output layer 1024 may generate a corresponding pressure value.

An example size of a regressor model is given in table 1026 of FIG. 10C.Column 1028 illustrates example layers, including the layers of FIG.10B. Column 1030 provides example output shapers of the layers 1028, andcolumn 1032 provides example numbers of parameters of each layer. Asshown in section 1034, about 7.6 k of free parameters may be sufficientto be trained and frozen offline, and used in production using scaledin-lab datasets. The resulting parameters may be stored in a permanentmemory of a device, such as the various example devices of FIGS. 5, 6,and 11-13B.

FIG. 11 is an example illustration of the system of FIG. 1, implementedusing earbud devices. In FIG. 11, an earbud 1102 and an earbud 1104 eachhave a passive electrode 1106 and a passive electrode 1108,respectively. The earbud 1102 also has an active electrode 1110, and theearbud 1104 has an active electrode 1112.

As referenced above, when wearing the earbuds 1102, 1104, the user 104may have each of the passive electrodes 1106, 1108 in a default state ofcontact with the user's ear. Therefore, pressing the active electrode1112 will complete a circuit with the passive electrode 1106 on theother side of the user's body, and may be used to perform a firstfunction, or set of functions. Similarly, pressing the active electrode1110 will complete a circuit with the passive electrode 1108 on theother side of the user's body, and may be used to perform a secondfunction, or set of functions.

FIG. 12 is an example illustration of the system of FIG. 1, implementedusing smartglasses 1202. In FIG. 12, passive electrode 1204 is locatedon the smartglasses 1202 to rest on, and maintain contact with, a bridgeof a user's nose. Then, an active electrode 1206 may be pressed tocomplete a circuit as described herein and obtain a first set offunctions. Additionally, another active electrode 1208 may be pressed tocomplete a circuit as described herein and obtain a second set offunctions.

In some implementations of both FIGS. 11 and 12, sufficient computingresources may be provided within the earbuds 1102, 1104 and thesmartglasses 1202 to implement some or all of the pressure detector 102of FIG. 1, and related hardware and software. In other implementations,the earbuds 1102, 1104 and/or the smartglasses 1202 may be in wired orwireless communication with another device having sufficient computingresources to implement some or all of the pressure detector 102 of FIG.1, and related hardware and software. For example, the earbuds 1102,1104 may be in communications with the smartglasses 1202, or either orboth of the earbuds 1102, 1104 and the smartglasses 1202 may be incommunication with a smartphone or other computing device.

FIG. 13A is an example illustration of the system of FIG. 1, implementedusing a smartphone 1302. That is, in addition to providing support forwearable devices such as the smartwatch of FIGS. 3-6, the earbuds ofFIG. 11, and the smartglasses of FIG. 12, the smartphone 1302 may beused to implement, and benefit from, the techniques of FIGS. 1 and 2.

In FIG. 13A, the smartphone 1302 is equipped with a first electrode 1304and a second electrode 1306. In this example, both the electrodes 1304,1306 are active electrodes, as both are in a default open circuitcondition. The user 104 may utilize the touch pressure inputs describedherein, for example, by holding the smartphone 1302 in a left hand andcontacting the first electrode 1304 with the left hand, while pressingthe second electrode 1306 with the right hand. Conversely, the same ordifferent set of features may be obtained by holding the smartphone 1302in a right hand and contacting the second electrode 1306 with the righthand, while pressing the first electrode 1304 with the left hand.

Although only the two electrodes 1304 and 1306 are illustrated in FIG.13, positioned on sides of the smartphone 1302, it will be appreciatedthat more than two electrodes may be used, and may be positionedvirtually anywhere on the smartphone 1302. Then, different combinations(e.g., pairs) may be used to implement different sets of userinteraction features.

FIG. 13B is an example illustration of the system of FIG. 1, implementedusing a keyboard 1308. In FIG. 13B, the keyboard 1308 is equipped with afirst electrode 1404 and a second electrode 1406. In this example, as inFIG. 13A, both the electrodes 1310, 1312 are active electrodes, as bothare in a default open circuit condition. The user 104 may utilize thetouch pressure inputs described herein, for example, by contacting thefirst electrode 1310 with the left hand, while pressing the secondelectrode 1312 with the right hand. Conversely, the same or differentset of features may be obtained by contacting the second electrode 1312with the right hand, while pressing the first electrode 1310 with theleft hand.

As with the smartphone 1302 of FIG. 13A, more than two electrodes may beused with the keyboard 1308, and may be placed anywhere desired on thekeyboard 1308. For example, electrodes may be applied at one or morekeys of the keyboard 1308, so that pressure-dependent functions may beassigned to different pairs of keys. For example, a shift key or controlkey may be assigned as a first electrode, and then pressing any otherkey with another electrode would complete a key-specific circuit toperform a function with respect to that key. For example, differentpressure levels could be assigned so that pressing a key with morepressure resulting in the corresponding letter being bolded, italicized,or underlined. In other implementations, shortcuts for various functionsmay be assigned to specific keys, and accessed by using assignedpressure levels. Thus, it will be appreciated that the techniquesdescribed herein for touch pressure input may be used in virtually anydevice, including peripheral devices, wearable devices, and combinationsthereof.

FIG. 14 shows an example of a computer device 1400 and a mobile computerdevice 1450, which may be used with the techniques described here.Computing device 1400 is intended to represent various forms of digitalcomputers, such as laptops, desktops, tablets, workstations, personaldigital assistants, smart devices, appliances, electronic sensor-baseddevices, televisions, servers, blade servers, mainframes, and otherappropriate computing devices. Computing device 1450 is intended torepresent various forms of mobile devices, such as personal digitalassistants, cellular telephones, smart phones, and other similarcomputing devices. The components shown here, their connections andrelationships, and their functions, are meant to be exemplary only, andare not meant to limit implementations described and/or claimed in thisdocument.

Computing device 1400 includes a processor 1402, memory 1404, a storagedevice 1406, a high-speed interface 1408 connecting to memory 1404 andhigh-speed expansion ports 1410, and a low speed interface 1412connecting to low speed bus 1414 and storage device 1406. The processor1402 can be a semiconductor-based processor. The memory 1404 can be asemiconductor-based memory. Each of the components 1402, 1404, 1406,1408, 1410, and 1412, are interconnected using various busses, and maybe mounted on a common motherboard or in other manners as appropriate.The processor 1402 can process instructions for execution within thecomputing device 1400, including instructions stored in the memory 1404or on the storage device 1406 to display graphical information for a GUIon an external input/output device, such as display 1416 coupled to highspeed interface 1408. In other implementations, multiple processorsand/or multiple buses may be used, as appropriate, along with multiplememories and types of memory. Also, multiple computing devices 1400 maybe connected, with each device providing portions of the necessaryoperations (e.g., as a server bank, a group of blade servers, or amulti-processor system).

The memory 1404 stores information within the computing device 1400. Inone implementation, the memory 1404 is a volatile memory unit or units.In another implementation, the memory 1404 is a non-volatile memory unitor units. The memory 1404 may also be another form of computer-readablemedium, such as a magnetic or optical disk. In general, thecomputer-readable medium may be a non-transitory computer-readablemedium.

The storage device 1406 is capable of providing mass storage for thecomputing device 1400. In one implementation, the storage device 1406may be or contain a computer-readable medium, such as a floppy diskdevice, a hard disk device, an optical disk device, or a tape device, aflash memory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods and/orcomputer-implemented methods, such as those described above. Theinformation carrier is a computer- or machine-readable medium, such asthe memory 1404, the storage device 1406, or memory on processor 1402.

The high speed controller 1408 manages bandwidth-intensive operationsfor the computing device 1400, while the low speed controller 1412manages lower bandwidth-intensive operations. Such allocation offunctions is exemplary only. In one implementation, the high-speedcontroller 1408 is coupled to memory 1404, display 1416 (e.g., through agraphics processor or accelerator), and to high-speed expansion ports1410, which may accept various expansion cards (not shown). In theimplementation, low-speed controller 1412 is coupled to storage device1406 and low-speed expansion port 1414. The low-speed expansion port,which may include various communication ports (e.g., USB, Bluetooth,Ethernet, wireless Ethernet) may be coupled to one or more input/outputdevices, such as a keyboard, a pointing device, a scanner, or anetworking device such as a switch or router, e.g., through a networkadapter.

The computing device 1400 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 1420, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 1424. Inaddition, it may be implemented in a computer such as a laptop computer1422. Alternatively, components from computing device 1400 may becombined with other components in a mobile device (not shown), such asdevice 1450. Each of such devices may contain one or more of computingdevice 1400, 1450, and an entire system may be made up of multiplecomputing devices 1400, 1450 communicating with each other.

Computing device 1450 includes a processor 1452, memory 1464, aninput/output device such as a display 1454, a communication interface1466, and a transceiver 1468, among other components. The device 1450may also be provided with a storage device, such as a microdrive orother device, to provide additional storage. Each of the components1450, 1452, 1464, 1454, 1466, and 1468, are interconnected using variousbuses, and several of the components may be mounted on a commonmotherboard or in other manners as appropriate.

The processor 1452 can execute instructions within the computing device1450, including instructions stored in the memory 1464. The processormay be implemented as a chipset of chips that include separate andmultiple analog and digital processors. The processor may provide, forexample, for coordination of the other components of the device 1450,such as control of user interfaces, applications run by device 1450, andwireless communication by device 1450.

Processor 1452 may communicate with a user through control interface1458 and display interface 1456 coupled to a display 1454. The display1454 may be, for example, a TFT LCD (Thin-Film-Transistor Liquid CrystalDisplay) or an OLED (Organic Light Emitting Diode) display, or otherappropriate display technology. The display interface 1456 may compriseappropriate circuitry for driving the display 1454 to present graphicaland other information to a user. The control interface 1458 may receivecommands from a user and convert them for submission to the processor1452. In addition, an external interface 1462 may be provided incommunication with processor 1452, so as to enable near areacommunication of device 1450 with other devices. External interface 1462may provide, for example, for wired communication in someimplementations, or for wireless communication in other implementations,and multiple interfaces may also be used.

The memory 1464 stores information within the computing device 1450. Thememory 1464 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 1484 may also be provided andconnected to device 1450 through expansion interface 1482, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 1484 may provide extra storage spacefor device 1450, or may also store applications or other information fordevice 1450. Specifically, expansion memory 1484 may includeinstructions to carry out or supplement the processes described above,and may include secure information also. Thus, for example, expansionmemory 1484 may be provided as a security module for device 1450, andmay be programmed with instructions that permit secure use of device1450. In addition, secure applications may be provided via the SIMMcards, along with additional information, such as placing identifyinginformation on the SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 1464, expansionmemory 1484, or memory on processor 1452, that may be received, forexample, over transceiver 1468 or external interface 1462.

Device 1450 may communicate wirelessly through communication interface1466, which may include digital signal processing circuitry wherenecessary. Communication interface 1466 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 1468. In addition, short-range communication may occur, suchas using a Bluetooth, low power Bluetooth, Wi-Fi, or other suchtransceiver (not shown). In addition, GPS (Global Positioning System)receiver module 1480 may provide additional navigation- andlocation-related wireless data to device 1450, which may be used asappropriate by applications running on device 1450.

Device 1450 may also communicate audibly using audio codec 1460, whichmay receive spoken information from a user and convert it to usabledigital information. Audio codec 1460 may likewise generate audiblesound for a user, such as through a speaker, e.g., in a handset ofdevice 1450. Such sound may include sound from voice telephone calls,may include recorded sound (e.g., voice messages, music files, etc.) andmay also include sound generated by applications operating on device1450.

The computing device 1450 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 1483. It may also be implemented as part of a smartphone 1481, personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as modules, programs, software,software applications or code) include machine instructions for aprogrammable processor, and can be implemented in a high-levelprocedural and/or object-oriented programming language, and/or inassembly/machine language. As used herein, the terms “machine-readablemedium” “computer-readable medium” refers to any computer programproduct, apparatus and/or device (e.g., magnetic discs, optical disks,memory, Programmable Logic Devices (PLDs)) used to provide machineinstructions and/or data to a programmable processor, including amachine-readable medium that receives machine instructions as amachine-readable signal. The term “machine-readable signal” refers toany signal used to provide machine instructions and/or data to aprogrammable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor,or LED (light emitting diode)) for displaying information to the userand a keyboard and a pointing device (e.g., a mouse or a trackball) bywhich the user can provide input to the computer. Other kinds of devicescan be used to provide for interaction with a user as well. For example,feedback provided to the user can be any form of sensory feedback (e.g.,visual feedback, auditory feedback, or tactile feedback), and input fromthe user can be received in any form, including acoustic, speech, ortactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

In some implementations, the computing devices depicted in FIG. 14 caninclude sensors that interface with, or are included in, a HMD 1490. Forexample, one or more sensors included on computing device 1450 or othercomputing device depicted in FIG. 14, can provide input to HMD 1490 orin general, provide input to that can be used by the HMD 1490. Thesensors can include, but are not limited to, a touchscreen,accelerometers, gyroscopes, pressure sensors, biometric sensors,temperature sensors, humidity sensors, and ambient light sensors.Computing device 1450 (e.g., the HMD 1490) can use the sensors todetermine an absolute position and/or a detected rotation of the HMD1490 that can then be used as input for use by the HMD 1490.

In some implementations, one or more input devices included on, orconnected to, the computing device 1450 and/or the HMD 1490 can be usedas inputs for use by the HMD 1490. The input devices can include, butare not limited to, a touchscreen, a keyboard, one or more buttons, atrackpad, a touchpad, a pointing device, a mouse, a trackball, ajoystick, a camera, a microphone, earphones or buds with inputfunctionality, a gaming controller, or other connectable input device.

In some implementations, one or more output devices included on thecomputing device 1450, and/or in the HMD 1490, can provide output and/orfeedback to a user of the HMD 1490. The output and feedback can bevisual, tactical, or audio. The output and/or feedback can include, butis not limited to, rendering a display of the HMD 1490, vibrations,turning on and off or blinking and/or flashing of one or more lights orstrobes, sounding an alarm, playing a chime, playing a song, and playingof an audio file. The output devices can include, but are not limitedto, vibration motors, vibration coils, piezoelectric devices,electrostatic devices, light emitting diodes (LEDs), strobes, andspeakers.

In some implementations, computing device 1450 can be placed within HMD1490 to create an integrated HMD system. HMD 1490 can include one ormore positioning elements that allow for the placement of computingdevice 1450, such as smart phone 1481, in the appropriate positionwithin HMD 1490. In such implementations, the display of smart phone1481 can render images using a display of the HMD 1490.

In some implementations, the computing device 1450 may appear as anotherobject in a computer-generated, 3D environment. Interactions by the userwith the computing device 1450 (e.g., rotating, shaking, touching atouchscreen, swiping a finger across a touch screen) can be interpretedas interactions with the object in the AR/VR space. As just one example,computing device can be a laser pointer. In such an example, computingdevice 1450 appears as a virtual laser pointer in thecomputer-generated, 3D environment. As the user manipulates computingdevice 1450, the user in the AR/VR space sees movement of the laserpointer. The user receives feedback from interactions with the computingdevice 1450 in the AR/VR environment on the computing device 1450 or onthe HMD 1490.

In some implementations, a computing device 1450 may include atouchscreen. For example, a user can interact with the touchscreen in aparticular manner that can mimic what happens on the touchscreen withwhat happens in a display of the HMD 1490. For example, a user may use apinching-type motion to zoom content displayed on the touchscreen. Thispinching-type motion on the touchscreen can cause information providedin display to be zoomed. In another example, the computing device may berendered as a virtual book in a computer-generated, 3D environment.

In some implementations, one or more input devices in addition to thecomputing device (e.g., a mouse, a keyboard) can be rendered in adisplay of the HMD 1490. The rendered input devices (e.g., the renderedmouse, the rendered keyboard) can be used as rendered in the in thedisplay.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the description and claims.

In addition, the logic flows depicted in the figures do not require theparticular order shown, or sequential order, to achieve desirableresults. In addition, other steps may be provided, or steps may beeliminated, from the described flows, and other components may be addedto, or removed from, the described systems. Accordingly, otherimplementations are within the scope of the following claims.

Further to the descriptions above, a user is provided with controlsallowing the user to make an election as to both if and when systems,programs, devices, networks, or features described herein may enablecollection of user information (e.g., information about a user's socialnetwork, social actions, or activities, profession, a user'spreferences, or a user's current location), and if the user is sentcontent or communications from a server. In addition, certain data maybe treated in one or more ways before it is stored or used, so that userinformation is removed. For example, a user's identity may be treated sothat no user information can be determined for the user, or a user'sgeographic location may be generalized where location information isobtained (such as to a city, ZIP code, or state level), so that aparticular location of a user cannot be determined. Thus, the user mayhave control over what information is collected about the user, how thatinformation is used, and what information is provided to the user.

The computer system (e.g., computing device) may be configured towirelessly communicate with a network server over a network via acommunication link established with the network server using any knownwireless communications technologies and protocols including radiofrequency (RF), microwave frequency (MWF), and/or infrared frequency(IRF) wireless communications technologies and protocols adapted forcommunication over the network.

In accordance with aspects of the disclosure, implementations of varioustechniques described herein may be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. Implementations may be implemented as a computerprogram product (e.g., a computer program tangibly embodied in aninformation carrier, a machine-readable storage device, acomputer-readable medium, a tangible computer-readable medium), forprocessing by, or to control the operation of, data processing apparatus(e.g., a programmable processor, a computer, or multiple computers). Insome implementations, a tangible computer-readable storage medium may beconfigured to store instructions that when executed cause a processor toperform a process. A computer program, such as the computer program(s)described above, may be written in any form of programming language,including compiled or interpreted languages, and may be deployed in anyform, including as a standalone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program may be deployed to be processed on one computer or onmultiple computers at one site or distributed across multiple sites andinterconnected by a communication network.

Specific structural and functional details disclosed herein are merelyrepresentative for purposes of describing example implementations.Example implementations, however, may be embodied in many alternateforms and should not be construed as limited to only the implementationsset forth herein.

The terminology used herein is for the purpose of describing particularimplementations only and is not intended to be limiting of theimplementations. As used herein, the singular forms “a,” “an,” and “the”are intended to include the plural forms as well, unless the contextclearly indicates otherwise. It will be further understood that theterms “comprises,” “comprising,” “includes,” and/or “including,” whenused in this specification, specify the presence of the stated features,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, steps, operations,elements, components, and/or groups thereof.

It will be understood that when an element is referred to as being“coupled,” “connected,” or “responsive” to, or “on,” another element, itcan be directly coupled, connected, or responsive to, or on, the otherelement, or intervening elements may also be present. In contrast, whenan element is referred to as being “directly coupled,” “directlyconnected,” or “directly responsive” to, or “directly on,” anotherelement, there are no intervening elements present. As used herein theterm “and/or” includes any and all combinations of one or more of theassociated listed items.

Spatially relative terms, such as “beneath,” “below,” “lower,” “above,”“upper,” and the like, may be used herein for ease of description todescribe one element or feature in relationship to another element(s) orfeature(s) as illustrated in the figures. It will be understood that thespatially relative terms are intended to encompass differentorientations of the device in use or operation in addition to theorientation depicted in the figures. For example, if the device in thefigures is turned over, elements described as “below” or “beneath” otherelements or features would then be oriented “above” the other elementsor features. Thus, the term “below” can encompass both an orientation ofabove and below. The device may be otherwise oriented (rotated 130degrees or at other orientations) and the spatially relative descriptorsused herein may be interpreted accordingly.

Example implementations of the concepts are described herein withreference to cross-sectional illustrations that are schematicillustrations of idealized implementations (and intermediate structures)of example implementations. As such, variations from the shapes of theillustrations as a result, for example, of manufacturing techniquesand/or tolerances, are to be expected. Thus, example implementations ofthe described concepts should not be construed as limited to theparticular shapes of regions illustrated herein but are to includedeviations in shapes that result, for example, from manufacturing.Accordingly, the regions illustrated in the figures are schematic innature and their shapes are not intended to illustrate the actual shapeof a region of a device and are not intended to limit the scope ofexample implementations.

It will be understood that although the terms “first,” “second,” etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another. Thus, a “first” element could be termed a“second” element without departing from the teachings of the presentimplementations.

Unless otherwise defined, the terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which these concepts belong. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and/orthe present specification and will not be interpreted in an idealized oroverly formal sense unless expressly so defined herein.

While certain features of the described implementations have beenillustrated as described herein, many modifications, substitutions,changes, and equivalents will now occur to those skilled in the art. Itis, therefore, to be understood that the appended claims are intended tocover such modifications and changes as fall within the scope of theimplementations. It should be understood that they have been presentedby way of example only, not limitation, and various changes in form anddetails may be made. Any portion of the apparatus and/or methodsdescribed herein may be combined in any combination, except mutuallyexclusive combinations. The implementations described herein can includevarious combinations and/or sub-combinations of the functions,components, and/or features of the different implementations described.

1. A computing device comprising: a processor; a storage medium storinginstructions; a body; and at least two electrodes mounted on the bodyand in contact with a user of the computing device, wherein theinstructions, when executed by the processor, cause the computing deviceto determine an electrical signal associated with a circuit thatincludes the at least two electrodes and the user, the electrical signalincluding an electrocardiography (ECG) signal characterizing electricalactivities of a heart of the user; filter the ECG signal to obtain afiltered signal in which components related to the electrical activitiesof the heart of the user have been removed; determine, from the filteredsignal, a pressure applied to at least one electrode of the at least twoelectrodes; and implement at least one function of the computing device,based on the pressure.
 2. (canceled)
 3. The computing device of claim 1,further configured to: determine, from the filtered signal, motionartifacts corresponding to the pressure applied by the user to the atleast one electrode; and determine the pressure, based on the motionartifacts.
 4. The computing device of claim 1, wherein the at least twoelectrodes include a passive electrode in default contact with the userduring use of the at least one computing device, and the at least oneelectrode includes an active electrode that receives a press from theuser to implement the at least one function.
 5. The computing device ofclaim 1, further configured to: determine the pressure as having atleast one pressure level from among a plurality of pre-defined pressurelevels, based on the electrical signal and at least one pre-determinedrelationship between electrical signals and the pre-defined pressurelevels.
 6. The computing device of claim 5, wherein the at least onepre-determined relationship includes a heuristic model.
 7. The computingdevice of claim 1, further configured to: access a machine learningmodel trained to classify electrical signal artifacts as correspondingpressure levels; and determine the pressure from among the pressurelevels, using the machine learning model.
 8. The computing device ofclaim 1, further configured to: determine the pressure as having apressure level from among a plurality of pre-defined pressure levels;and implement the at least one function including selecting or operatingthe at least one function in response to the pressure level.
 9. Thecomputing device of claim 1, further configured to: determine thepressure as having a pressure level from among a plurality ofpre-defined pressure levels; and generate a feedback signal indicatingthe pressure level.
 10. The computing device of claim 1, wherein thecomputing device includes at least one wearable device, and the at leasttwo electrodes include a passive electrode that is in default contactwith a skin surface of the user while the at least one wearable deviceis being worn, and the at least one electrode includes an activeelectrode that is accessible to the user to receive the pressure appliedthereto by the user.
 11. The computing device of claim 1, wherein the atleast two electrodes include at least three electrodes, the computingdevice further configured to: determine a second electrical signal atthe computing device, the second electrical signal associated with asecond circuit that includes a different electrode combination of the atleast three electrodes than the circuit; determine, from the secondelectrical signal, a second pressure applied to at least one electrodeof the different electrode combination; and implement at least a secondfunction of the computing device, based on the second pressure.
 12. Acomputer-implemented method for operating a computing device having aprocessor, a storage medium storing instructions executable by theprocessor to perform the computer-implemented method, and a body with atleast two electrodes mounted thereon, the method comprising: determiningan electrical signal at the computing device, the electrical signalassociated with a circuit that includes the at least two electrodes incontact with a user of the computing device, and the user, theelectrical signal including an electrocardiography (ECG) signalcharacterizing electrical activities of a heart of the user; filter theECG signal to obtain a filtered signal in which components related tothe electrical activities of the heart of the user have been removed;determining, from the filtered signal, a pressure applied to at leastone electrode of the at least two electrodes; and implementing at leastone function of the computing device, based on the pressure. 13.(canceled)
 14. The method of claim 12, further comprising: determining,from the filtered signal, motion artifacts corresponding to motions ofthe user in applying the pressure to the at least one electrode; anddetermining the pressure, based on the motion artifacts.
 15. The methodof claim 12, further comprising: determining the pressure as having atleast one pressure level from among a plurality of pre-defined pressurelevels, based on the electrical signal and at least one pre-determinedrelationship between electrical signals and the pre-defined pressurelevels.
 16. The method of claim 15, wherein the at least onepre-determined relationship includes a heuristic model.
 17. The methodof claim 12, further comprising: determining the pressure as having apressure level from among a plurality of pre-defined pressure levels;and implementing the at least one function including selecting oroperating the at least one function in response to the pressure level.18. A computer program product, the computer program product beingtangibly embodied on a non-transitory computer-readable storage mediumand comprising instructions that, when executed by a computing device,are configured to cause the computing device to determine an electricalsignal at the computing device, the electrical signal associated with acircuit that includes at least two electrodes mounted on a body of thecomputing device and in contact with a user of the computing device, andthe user, the electrical signal including an electrocardiography (ECG)signal characterizing electrical activities of a heart of the user;filter the ECG signal to obtain a filtered signal in which componentsrelated to the electrical activities of the heart of the user have beenremoved; determine, from the filtered signal, a pressure applied to atleast one electrode of the at least two electrodes; and implement atleast one function of the computing device, based on the pressure. 19.The computer program product of claim 18, wherein the instructions, whenexecuted, are further configured to cause the computing device to:determine, from the filtered signal, motion artifacts corresponding tomotions of the user in applying the pressure to the at least oneelectrode; and determine the pressure, based on the motion artifacts.20. The computer program product of claim 18, wherein the instructions,when executed, are further configured to cause the computing device to:determine the pressure as having a pressure level from among a pluralityof pre-defined pressure levels; and generate a feedback signalindicating the pressure level.